Problems with Listening to Algebra: Some Design Issues

Thework reported here was part of research into the design of theMathtalk program. This allows visually disabled people to read algebra notation using synthetic speech andnon-speech audio [7]. This paper describes an experiment that investigated the problems of reading algebra notation by listening. Themain theme is the use of externalmemory in the readingprocess and the difficulties this implies for visually disabled users. In addition, the possibility of using prosodic cues to facilitate parsing of the spoken presentation was investigated. Externalmemory is a valuable resource inmathematical problem solving [3]. Mathematicians rely on pencil and paper to supplement verbal communication. Imagine listening to a lecture on algebra without any visual aids for support. The printed representation of mathematical concepts is thus vital for communication, not only between mathematicians, but for the single user for whom graphical representations such as a text-book page act as an external memory. Listening to algebra is difficult because expressions written in algebraic notation can be complex and syntactically sensitive. Losing one element of content or structure may drastically alter meaning. A graphical representation relieves the reader of the burden of retaining all the information inworkingmemory. The spatial layout of an expression provides cues to structure and facilitates the parsing of the notation [4]. In addition the visual form supports browsing and rereading. For listening readers this form of external memory does not exist; the speech signal has to be retained or it is lost to the listener. Thus a greater burden is placed on the working memory resources of the listener and items are easily lost froman expression. This situation ismadeworse when synthetic speech is used since cues such as prosody and intonation are absent. In addition a listening reader cannot review the information on the external memory rapidly. Because such facilities are not available to the listener, reading becomes a passive, not an active process, and review of information becomes slow and tedious. The overall aim of the Mathtalk program is to make listening active through the use of improved synthetic speech and through auditory analogues of visual external memory. One aim of the research reported here was to discover whether the addition of a prosodic element could facilitate the parsing and retention of spoken algebra expressions. A speakermay use rhythm, pitch tempo and amplitude to drastically alter the information content of utterances that contain the same words. The feature of prosody useful to the presentation of algebra notation